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Abstract 


There are many tools that systems engineers use in today’s space programs. In my time in the Commercial Crew 
Program I sought to improve one of the vital tools for the verification and validation team. This was my main project 
but only a small part of what I have done in the department. I have also had the chance to learn from the best and see 
actual hardware, this real world experience will help me be a better aerospace engineer when I enter the workforce. I 
look forward to seeing the Commercial Crew Program progress to launch. 


Nomenclature 
VE = Verification event 
CCP = Commercial Crew Program 
SE&I = Systems Engineering and Integration 
V&V = Verification and Validation 
RO = Requirement Owner 


I. Introduction 


The Commercial Crew Program was created to enable private companies to carry NASA astronauts to the 
International Space Station. The job of designing, testing, building, operating, and maintaining the hardware, software 
and ground support falls on the Partner. These Partners are paid when that meet contractual milestones. NASA’s 
primary job in the Program is to provide technical expertise, and to validate that the Partners are meeting the 
contractual requirements. NASA functions more as a regulatory agency than the owner of the system. This is a big 
change from previous programs. There is a strict certification process that the Partner must meet before they are 
allowed to transport humans, as well as the requirements needed for the ISS Program. These requirements are wrapped 
up in to the 1100 requirement set. Each Partner has come up with a plan to complete all of the requirements. The SE&I 
team works with the Partner and the NASA engineers to coordinate the closure of Verification Events, and adherence 
to the requirements. A VE is a test, demonstration, inspection, or analysis that shows adherence to a requirement or 
multiple requirements. 


In the field of aerospace engineering many projects are very large. Elements are broken into systems and 
subsystems which are built up from components. The systems must all work together. Issues can arise when one part 
of the system does not communicate well with the rest of the system. The job of systems engineers are to integrate 
different teams. The sub-set of the SE&I team that I work in is the Verification and Validation team. We work to 


ensure that the Partner is meeting all of the requirements in 1130, if they meet the requirements that is considered 
verifications. Verification is defined as “an evaluation of the performance of the as-designed and as-built end-item 
with respect to defined requirements” (Space Vehicle Test and Evaluation Handbook). 


II. Project 


The first task that I had when I came on board was to get up to speed. To do this I read the certification plan from 
the Partner. Then to compare, I read a portion of the Space Vehicle Test and Evaluation Handbook to see how 
certification was previously done. It became apparent that there are many ways to do it. From there I read up on 
previous failures in systems engineering. The process that we now go through was heavily shaped by the lessons 
learned from previous programs. NASA now functions much differently than it did in the shuttle years. Back in the 
days of the space shuttle all of the data was delivered by hand from the Partner, and all of the requirement work was 
done by hand. Now there are databases that allow all of the people that need to sign off on a requirement to do so from 
any NASA center. Being that the Commercial Crew Program is spread out over the country, this is essential. The VE 
tracker was created to act as a quick and organized way to show everyone in the Program the status of all of the VE’s. 
PowerPoint presentations and excel spreadsheets have come a very long way now we can have an automatically 
updated spreadsheet that can be put into a PowerPoint that many people can work on at the same time, and then be 
presented on a large screen to aid the proceedings of the meeting and the explanation of metrics. This is especially 
true when a complex issue with a lot of data needs to be discussed. 


The importance of tools for integration can’t be over stated. Without the use of cloud-type databases it would be 
much harder and much slower to have a large group of people edit and change a document. There are many tools used 
now that previous NASA programs did not have. Now when data is delivered for the partner many people have access 
and nothing is stored on a single computer. SharePoint acts the hub for all the activity, allowing the simultaneous 
editing and updating of complex documents. This amount of simultaneous collaboration allows a much smaller team 
cross country team to accomplish a review in a short period of time with no travel costs. These new tools are also 
deigned to show accountability, so nothing can slip through he cracks. This means a safer flight, and more efficient 
Program. 


The next step in getting me up to speed was to go to a lot of meetings and just listen to the conversation. There were 
a lot of words and lots of acronyms that I did not understand. But with a lot of questions I was able to understand what 
they were talking about. Now I was prepared to begin work on my project. The task given to me for the Internship 
was to improve an excel spreadsheet, this spreadsheet is shared with program management, the working team and in 
some cases the Partners. This spreadsheet is primarily a status tracker for all of the VEs and the 1130 requirements 
they are linked to. 


The excel spreadsheet that I added to acts as a front page for all of these verification events. There is a lot of things 
going on behind the scenes. Some tests apply to 15 to 20 requirements, each having multiple people signing off. Some 
of these requirements need special procedures. Some requirements require coordination with the ISS team. The ISS 
integration team works to their own requirements documented in SSP 50808. Some tests that the Partner does attempts 
to fulfill both 1130 requirements and SSP 50808 requirements. As part of my task I had to create a way for the excel 
spreadsheet to identify what verification events had requirements that needed coordination between Commercial Crew 
office and the ISS verification team. 


This spreadsheet pulls from multiple databases to create a system that tracks the status of the pending and upcoming 
VEs. Some of the tasks that needed to be complete required the use of visual basic. Visual basic is a programing 
language that allows manipulation of excel and other Microsoft office products. I did some research of Visual basic 
on my own. I also talked to the expert on visual Basic and the creator of the original spread sheet to improve my skills. 
I also had to think outside the box when it came to creating the addition of three columns. This was a very good 
exercise to learn more of the excel commands and build upon my knowledge. I used these new skills to add two new 
columns to the VE tracker. These columns gave more information on the event, whether it needs special engineering 
approval, and/or the VE to be coordinated with the ISS Program. 


As part of the systems engineering and integration team we deal with issues that involve many people. Many of 
the issues are also very complex. With the addition of the three new columns it very clearly lays out if there are special 
steps that need to be taken to close the verification event. This helps to facilitate the closure of large and complex 
verification events. There is also an issue of finding a way to quickly update management in the progress, and when 
progress has stalled. With that in mind, I coordinated with the creator of the excel spread sheet to create a counter to 
identify those verification events that had disconnects between the Partner and NASA. To quickly show the important 
information the tracker highlighted the amount of time this VE has been in discussion. Those that have had lots of 
back and forth require attention of the reader. This highlighting saves time when identifying issues with a verification 
event. I also added a column that would allow the filtering of verification events to those that required a lot of offices 
to work together. The filtering of results is also a good way of getting people to focus on the topic and quickly glean 
the intended purpose. The filtering also identifying the group or groups that may require more integration before the 
event can be shown as closed. 


I was given a list of the requirements, but to integrate them was a difficult task. The first few days on the task it 
took more of an exploratory approach. It was apparent that a manual input would not be practical as there are over 
300 VEs. The first approach used was a VLOOKUP table while it initially worked. It became apparent that the 
VLOOKUP table failed to identify certain VEs as a shared requirement. After analyzing the failed cases I showed the 
table was only successful when the requirement listed first was a shared requirement. The solution of a VLOOKUP 
table was therefore not successful. The next solution tried was using a large table to use a FIND command, this creates 
its own problems though this command only outputs the number of characters it takes in the cell to find the requirement 
it matches. Luckily there is a column pulled for one of the databases that lists out all of the requirements. So this 
command goes through each cell filled with every requirement tied to that verification event. When it finds the exact 
requirement, it notes the characters. Then another table went through and if the cell contained a number it is replaced 
requirement ID in the NASA database. These requirement ID’s are then concatenated in to the final cell that is display 
all the requirement ID’s. The final step was to add formatting that highlighted all the cells that had at least one 
requirement that was a shared ISS requirement. This highlighting is very important because it identifies quickly those 
VE’s that are shared 


Another column created was to identify the VEs that had requirements tied to them that required special 
engineering approval. These VEs require a more in-depth review from NASA. It is important to note the tests that 
require special approval long beforehand so the process is not held up in the short time period that NASA has to review 
the VE and accept it. The process for creating this column was very similar to the ISS coordination column. I pulled 
a list, made a FIND table, and then substituted the number with the exact requirement ID and then concatenated them 
into the final cell for each VE. Then highlighted the cells that had at least one requirement that was as special 
engineering approval 


A third column was added to make the reading of the spreadsheet easier. The column shows the number of offices 
that had a responsible owner for a specific VE. The RO is the leader for the requirement, but as noted above some 
VE’s are used to meet lots of requirements. This problem is easily solved if all of the RO’s are in the same System 
offices. That System office can integrate internally without SE&I help. In some VE’s there are multiple requirements 
leading to RO’s from more than two system offices. These VE’s require pre-coordination and one of those RO’s 
should be the lead for that VE or the SE&I office will acts as the lead integrator. The excel spreadsheet has a search 
function so the column also acted as a filter to show what VE’s need pre-coordination. The process for creating this 
column was similar to the other columns but with some differences. One being the cell searched is the RO Cell, and 
the other being that the number was more important than actual office. So instead of replacing FIND return with the 
requirement it was replaced by the number | then at the end they were summed. The resulting summation column was 
then added to the main sheet. The column then was conditionally formatted so that all the cells with a number greater 
than two. 


To make the spreadsheet more legible in presentations. The Signatories was compacted and the names were 
organized by their department. This task was done with the help for the excel expert. To accomplish this task visual 
basic was used. Another column that was added was a counter of status changes. This column also need to be made 
in visual basic. This column is of particular importance as it shows what VE’s have disconnects. These are then brought 
up at meetings with the Partners. 


Engr - Dunn, Kevin; Kunkle, John; 
‘Butler, Marc; Gaylor, Kent; Peacock, 


R.CTS.351 Command and Telemetry 
Communications 
R.CTS.380 Communication 
R.CTS,002 Simultaneous Operation of 
Spacecraft 
R.CTS.095 Communications with CVCC 
after Landing 
R.CTS.113 Voice Communication with 
Crew 
R.CTS.116 Private Audio Safety - Merrill, Richard; Sickler, 
R.CTS.038 Isolate and Recover from Leigh; Goebel, Tom 
SE&I - McKinnie, John; Chan, Steve; 
Peters, Jim 
Spacecraft - Randall, David; Quince, 


Engr - Bue, Grant; Pham, Chau; 
Moeller, Chad 
FOD - Moncla, Brandon (SpaceX); 


R.CTS,.094 Crew Survival Kit 
Safety - Fortson, Russ; Alberts, Seth 


Spacecraft - Daniel, Brian (SpaceX); 


R.CTS.O11 Return Crew 
R.CTS.373 Sea State Landing Limits for 
Aborts 


R.CTS.318 Medical Hardware Interfaces = . Engr - Williams, Dave 
Brian; Daniel, Brian FOD - Matthew, Christie 
(Spacex) H&P - Baitland, Lawrence 


R.CTS.107 Privacy of Health and Medical GMO - Hudson, David FOD - Koch, Will (SpaceX) 
Data (SpaceX) GMO - Hudson, David (SpaceX) 


8&1 - Chan, Steve 
R.CTS. 109 Health Stabilization GMO - Payne, Steve FOO - Moncla, Brandon (SpaceX) 
GMO - Payne, Steve 
HH&P - Haralson, Rhonda 


FOO - Owens, David; Spencer, Ron 

(Spacex) 

GMO - Hudson, David (SpaceX) 

HH&P - Haralson, Rhonda 

ISS - Simon, Tom (SpaceX) 
R.CTS.351 Command and Telemetry GMO - Hudson, David Engr - Gaylor, Kent 
Communications 0 OD - Owens. David 


Figure 1. Verification Event fracker’ showing the columns that were added, all sensitive partner data had been 
scrubbed 


In my time here at KSC the excel spreadsheet has only taken up a portion of my time the rest has been spent being 
a member of the Verification and Validation team, I have helped with various projects. I have attended many of the 
meetings that the team has had. This exposure to an actual systems engineering and integration process has taught me 
a lot. I took a systems engineering class at my university. While this class showed me a broad overview of the process, 
my experience here has given me an up close view of a program. I have learned that it is never simple, and the larger 
the project the harder it is. I was given the opportunity to see hardware of the Partner. This has given me a great 
appreciation for the pace of the Program, the building of the hardware is extremely meticulous as is necessary when a 
small flaw can lead to catastrophic results. I also have experienced how a large project is managed (timetable, risk, 
and cost). This is interesting and something that is not normally taught to engineers. 


Software is often overlooked in the construction of a spacecraft. It may appear as though one can simply construct 
the craft and then develop the software as an afterthought. Many of those in the SE&I team understand that this is 


simply not the case. Hardware and software must be designed in tandem and both must be integrated into the spacecraft 
to ensure a timely and safe launch. To make sure software was integrated properly I went through all of the 
requirements to identify all those tests and demonstration that use flight-like or flight software. First I narrowed down 
the list by removing those that have a primary stakeholder who is software focused person. Next I scoured the list of 
secondary stakeholders to remove those that have a secondary stakeholder that is software focused. With the list 
narrowed down significantly, I moved on to looking at what VEs these requirements are ties to. The nature of how 
these VEs are signed off means that as long as one requirement in a VE has a software stakeholder on one of the 
requirements that would be ok. This is because someone that signs of signs off for the total VE, not for a specific 
requirement. So I filtered out all of the requirements that were always tied to VEs that have requirements that have a 
software stakeholder. 


Another task given to me was to help setup a PowerPoint presentation for the CCP manager to present to a board 
of all the center directors and the NASA director at headquarters. This meeting is a yearly update to show the progress 
to the Program. For my task I started with the presentation from the previous year. So I first scrubbed the data from 
the previous year leaving only the template. I then went through the previous years and put together a draft for the 
slides from the previous 4 quarter progress reports. The draft was then brought up to the manager of CCP to get her 
thoughts and amendments. A lot of my time was spent finding relevant pictures for the tests and hardware updates for 
each Partner. I also spent time meeting with the certification leads for each Partner to ensure that the main highlights 
from tests and hardware milestones were presented in the meeting. I also met with those in charge of the metrics. The 
purpose of these meetings were to fact check the numbers presented in the slides. Another objective for the meetings 
were to determine the most important uncompleted items from the past year. This activity really helped me get a feel 
for the progress the Program has made over the past year. Below are some of the pictures that I pulled together. These 
show various verification and qualification tests. 
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Parachute Test 


Merlin 1D Full Thrust Testing 


Figure 2. An excerpt from the annual review charts put together, also showing actual verification events. 


I have received a lot of exposure to the manufacturing of spacecraft. I have been in many meetings where this 
hardware is being discussed. I was able to tour the building where one of the Partner’s is building their spacecraft. 
Another opportunity I received was to tour the launch pad of another Partner. The launch pad was the historic LC- 
39A. There have been over 100 launches from this pad including all of the moon landings, the first and last shuttle 
launches, and is now being utilized by a private company to launch cargo to the ISS. It was very interesting to see an 
active Launchpad and all of the hardware that is there. Seeing all of the actual hardware puts all of the integration 
work, and all of the lessons that I have learned in school into a real world perspective. 


I was also tasked to an in-depth look at the VEs that will be closed in the coming months. This was a rather tedious 
task that involved going into the NASA database and pulling out all of the data that relates to these VEs. The data that 
was pulled out was placed in a table. The table was broken down by system office. Each requirement needing to be 
closed for this VE is listed under what office owns that requirement. This table was displayed in PowerPoint charts 
for easy explanation of complex VEs. The RO for each requirement was listed as well as the signatories for each 
individual requirement. In addition to all of the 1130 requirements, I also added the ISS requirements that are also 
being accomplished for the VE. The picture this expanded view gives really helps the systems offices coordinate these 
complex VEs. 


My experience here in CCP has taught me why systems engineering and the work of integration is vital. Because 
space travel is inherently risky, there is no way that the risk can be zero, but the question that this Program deals with 
is how much risk can be accepted. The loss of crew risk is a balancing act between being too stringent and making the 
mission overly cumbersome, and putting the crew at unnecessary risk. . The end goal of the systems engineering and 
integration it to lower the risk by ensuring that all of the parts are communicating and that one knows what the other 
is doing. In our smaller group the V&V team works with the System offices to make sure the contract requirements 
are met team. The V&V team also creates tools to help them get a quick and easy visual to help the teams in the 
integration process. 


The Commercial Crew Program is the first of its kind. NASA is acting more like a regulatory agency than the 
builder. While in the past it was not uncommon for NASA to subcontract out parts of the mission, in this program 
NASA has given the task of designing, testing, operating, and maintaining entirely to a company and not the 
government. This has led to some interesting issues. NASA is still learning how to take a backseat. While other NASA 
programs have successfully taken cargo to the ISS, manned missions are significantly more complicated and require 
more safety and oversite. 


III. Conclusion 


In conclusion I have taken on and completed many tasks for the CCP office. I have also learned many things about 
the systems engineering process. These lessons that I have learned and the actual hardware that I have seen are 
invaluable to my education. The main project that I had for the internship was the excel spreadsheet. This serves as a 
vital tool for the personnel in the SE&I team. I have helped improve a tool to allow the V&V team to quickly 
communicate with the rest of the team and show more information about certain VEs. This is especially true for those 
VEs that have a large amount of data or have many requirements and system offices and therefor require a large 
amount of integration. I have also taken on tasks ranging from verifying that all VEs calling out software have a 
software stakeholder to helping build a PowerPoint for the CCP manager to give to the NASA director. This internship 
has given me to see the production of spacecraft, active launch-pad, and multiple commercial launches from the 
spaceport. All of these activities have helped further my education as an aerospace engineer. The actual integration of 
space systems is something not easily taught in classes. I also learned a great deal from the exposure I had to some of 
the philosophical discussions that accompany working in a manned space program. I have had a great time in Florida 
and when I complete my studies I look forward to working in the space industry. 
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